Bi-directional probing of software

ABSTRACT

Method and system are disclosed for bi-directional probing of software. The bidirectional probe is capable of transferring data to and from a software under test. This two-way transfer of data allows the variables and arguments in the software to not only be monitored, but also changed as needed. Test vectors may be developed and inserted into the software while running for testing purposes. Regression analysis may be made easier by using data from previous iterations as input for the next iterations.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application which claims the benefitof U.S. patent application Ser. No. 10/428,733 filed on May 1, 2003,which claims the benefit of U.S. Provisional Patent Application No.60/412,834 filed on Sep. 23, 2002, the disclosure of which is fullyincorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The invention is related to testing of software and, in particular, to amethod for bidirectional probing of software.

2. Description of the Related Art

Among developers of software, one of the most important requirements isfor the software to be reliable. Reliability refers to the ability of asoftware to operate without failure for a specified amount of time in aspecified environment. To ensure a sufficiently high level ofreliability, software must be thoroughly tested and debugged prior torelease. Usually, the entire software program as a whole is tested, aswell as the individual functional components (e.g., function calls,subroutines) that make up the software program. Typically, test vectorsare generated containing a series of values for the variables that arerequired by the software and/or one or more functional componentsthereof. The variable values are chosen to represent various types ofusage conditions and environments in which the software is intended tobe run. The test vectors are then applied to the software and/or the oneor more functional components thereof, and the variable values areobserved and recorded.

One type of testing that is often performed is called regressionanalysis, or sometimes verification testing. Regression analysisinvolves the selective retesting of a software that has been modified inorder to fix known problems. The selective retesting is performed inorder to ensure that the identified problems have been fixed, and thatno other previously working functional components have failed as aresult of the reparations. This type of testing is basically a qualitycontrol measure to ensure that the modified code still complies with itsspecified requirements and that any unmodified code has not beenaffected by the maintenance activity.

An important feature in regression analysis specifically and in softwaretesting in general is the ability to observe the variable valuesresulting from the test vectors. Early attempts to observe the variablevalues of a software and/or the functional components thereof involvedmanually setting break points and other traps in the source code itself.More recently, software development tools such as Code Composer Studio™from Texas Instruments and LabVIEW™ from National Instruments includesoftware probes that may be inserted into the code under test. Thesoftware probes allow the variables in the code under test to beobserved in real-time as the software is executed. These lattersolutions, however, are based only on getting the variable values outfrom the code under test (e.g., so they can be analyzed). They do notallow the variable values to be changed during the execution of thesoftware. In other words, presently existing software probes are onlyone-way or unidirectional probes in that the data is allowed to flowonly from the code under test to the test system. They do not allow thedirection of data transfer to be reversed so that data flows from thetest system into the code under test.

Accordingly, it would be desirable to provide a way to probe software ina manner such that data may be transferred both out of as well as intothe code under test.

SUMMARY OF THE INVENTION

Briefly, the present invention is directed to bi-directional probing ofsoftware. The bi-directional probe of the present invention is capableof transferring data to and from a software under test. This two-waytransfer of data allows the variables in the software to not only bemonitored, but also changed as needed. Test vectors may be developed andinjected into the software while running for testing purposes.Regression analysis is made easier by using data from previousiterations as input for the next iterations.

In general, in one embodiment, the invention is directed to a method oftesting software having a plurality of data variables and functionarguments therein. The method comprises executing the software,identifying an address location for at least one of the variables orarguments used by the software, and outputting any data stored in theaddress location to a test system to thereby monitor the data. Data fromthe test system is then inputted into the address location to therebyreplace any data previously stored in the address location. In general,in another embodiment, the invention is directed to an apparatus fortesting software having a plurality of data variables and functionarguments therein. The apparatus comprises a central processing unit,and a storage unit connected to the central processing unit. The storageunit stores computer readable instructions for instructing the centralprocessing unit to execute the software, identify an address locationfor at least one of the variables or arguments used by the software,output any data stored in the address location to the central processingunit to thereby monitor the data, and input data from the centralprocessing unit into the address location to thereby replace any datapreviously stored in the address location.

In general, in yet another embodiment, the invention is directed to asystem for testing software having a plurality of data variables andfunction arguments therein. The system comprises a device under testconfigured to execute the software including one or more probeinstructions in the software, and a tester connected to the device undertest. The tester is configured to control the device under test so thatwhen a probe instruction is executed, the device under test: willidentify an address location for at least one of the variables orarguments used by the software; output any data stored in the addresslocation to the tester; and input data received from the tester into theaddress location.

It should be emphasized that the term comprises/comprising, when used inthis specification, is taken to specify the presence of stated features,integers, steps, or components, but does not preclude the presence oraddition of one or more other features, integers, steps, components, orgroups thereof.

BRIEF DESCRIPTION OF THE DRAWINGS

A better understanding of the invention may be had by reference to thefollowing detailed description when taken in conjunction with theaccompanying drawings, wherein:

FIG. 1 illustrates an exemplary software testing environment accordingto embodiments of the invention using analogous hardware components;

FIGS. 2A-2D illustrates exemplary operating modes of the bi-directionalsoftware probe according to embodiments of the invention using analogoushardware components;

FIG. 3 illustrates an exemplary system in which the bi-directionalsoftware probe according to embodiments of the invention may beimplemented;

FIG. 4 illustrates another exemplary system in which the bi-directionalsoftware probe according to embodiments of the invention may beimplemented; and

FIG. 5 illustrates an exemplary method of implementing thebi-directional software probe according to embodiments of the invention.

DETAILED DESCRIPTION OF THE INVENTION

Following is a detailed description of the invention with reference tothe drawings wherein reference numerals for the same and similarelements are carried forward.

Embodiments of the invention provide a method and system for testingsoftware using bi-directional probes. The bi-directional probes of theinvention may be inserted into the program code at essentially anylocation. The probes allow data to be captured from as well as injectedinto the software. Specifically, the probes allow the values of thevariables in the software to be monitored, changed and inserted backinto the software during execution. The software is then furtherexecuted with the changed values. The bi-directional probes of theinvention may be implemented as a feature or a function in a softwaredevelopment tool such as Code Composer Studio™ from Texas Instrumentsand LabVIEW™ from National Instruments, or other similar softwaredevelopment environments.

The bi-directional software probing technique of the present inventionis somewhat analogous to the testing of a hardware circuit board.Therefore, the invention will be described initially in terms of a testsystem for a hardware circuit board. This description is provided forillustrative purposes only, however, as the invention is actuallydirected to the probing of software.

FIG. 1 illustrates a hardware test system 100 that is analogous to thesoftware testing tool in which the bi-directional probing technique ofthe present invention may be used. The hardware test system 100 isconnected to a device under test (DUT) 102 via a plurality of hardwareprobes, one of which is indicated at 104. Each hardware probe 104 may beidentified by its probe ID. For example, the first probe is PID 1, thesecond probe is PID 2, the third probe is PID 3, and so on. The probes104 are connected to one side of a cross-circuit box 106, the other sideof which is connected to one or more function generators 108, such aswaveform generators, and one or more measurement units 110, such asoscilloscopes and wavemeters. The cross-circuit box 106 allows theprobes 104 to be selectively connected to and disconnected from thefunction generators 108 and the measurement units 110 of the test system100. A controller (not expressly shown) in the test system 100 providesa control signal that controls the connectivity of the cross-circuit box106.

As can be seen, the probes 104 are strategically placed in order toallow the electrical signals at certain points of interest on the DUT102 to be probed. For example, the first probe PID 1 is placed at theinput of Func2 in order to allow electrical signal “a” to be probed.Likewise, the second probe Pill 2 is placed at the input of Func1 inorder to allow electrical signal “b” to be probed. The fifth probe PID5, however, is placed at the output of Func1 in order to allowelectrical signal “d” to be probed. The various functions (i.e., Func1-3) may be any function that can be performed by the DUT (e.g., adding,subtracting, averaging, etc.). Some functions may have one or moreinternal and/or sub-functions therein that may also be probed. Forexample, Func3 has a sub-function “f” included therein that may beprobed. In a manner similar to that described, the bi-directional probeof the present invention allows certain variables of interest in thesoftware to be probed.

The connection point of each probe 104 to the DUT is analogous to atypical wired pair connection, shown in the dashed circle indicated by112. As can be seen, one wire 114 of the wired pair 112 leads from theDUT 102 to the cross-circuit box 106, while the other wire 116 of thewired pair 112 leads from the cross-circuit box 106 back to the DUT 102.Similarly, the connection point of each probe 104 to the cross-circuitbox 106 is analogous to a pair of switches, shown in the dashed circleindicated by 118. The inbound switch, indicated at 120, selectivelyconnects the incoming wire 114 of a probe 104 to the tester system 100(e.g., to a measurement unit). The outbound switch, indicated at 122,selectively connects the return wire 116 of a probe 104 to either theincoming wire 114 (e.g., for normal operation) or to the tester system100 (e.g., to a function generator). The various modes of operation ofthe switches will be described in more detail below.

Referring now to FIGS. 2A-2D, the basic operating modes of the switchesthat connect the probes to the cross-circuit box are shown. Theseoperating modes graphically illustrate the functional capability of thesoftware probe of the present invention. In the first operating mode,shown in FIG. 2A, the inbound switch 120 and the outbound switch 122 areboth disconnected from the cross-circuit box 106 and are connected toeach other instead. This is the normal operating mode where data isneither flowing from the DUT 102 into the test system 100 or from thetest system 100 into the DUT 102. In the second operating mode, shown inFIG. 2B, the inbound switch 120 connects the DUT 102 to the test system100 while the outbound switch 122 is still connected to the inboundswitch 120 (i.e., disconnected from the test system). This operatingmode is used in order to obtain data from the DUT 102 for monitoringpurposes. In the third operating mode, shown in FIG. 2C, the outboundswitch 122 is connected to the test system 100, while the inbound switch120 is disconnected from the test system 100. This operating mode isused for injecting data from the test system 100 into the DUT 102 fortesting purposes. In the fourth operating mode, shown in FIG. 2D, boththe inbound switch 120 and the outbound switch 122 are connected to thetest system 100. This operating mode is used to obtain data from the DUT102 for monitoring purposes as well as for inserting data into the DUT102 for testing purposes. In a similar manner, the bi-directional probeof the present invention may be used to obtain data from the variablesand arguments of a software program under test, input data into thesevariables and arguments, or both.

An exemplary block of programming code containing bi-directional probeinstructions according to embodiments of the invention is shown inExample 1 below. As can be seen, the block of programming code iswritten in pseudocode and not in any particular programming language inorder to emphasize the generic nature and applicability of thebi-directional probe. In the example, Func0 is the code under test andis analogous to the DUT 102 of FIG. 1. The “probe” instructions areanalogous to the hardware probes PID 1-5 of FIG. 1, and typicallyinclude a probe ID as well as an indication of the variable or argumentto be probed as arguments. For example, “probe(1,c)” refers to the firstprobe PID 1, and affects the address location corresponding to thevariable “c” in the code under test. Thus, the probe instruction“probe(1,c)” allows the variable “c” in the block of programming code tobe monitored and changed as needed. Likewise, the probe instruction“probe(4,a)” allows the variable “a” to be monitored and changed asneeded, and so on.

Example 1

Func0(a,b,c) { probe(1,c) probe(2,b) probe(4,a) d = funcl(b) probe(5,d)probe(3,g) e,f= func2(c,g) probe(6,e) h,g = func3(a,d,e,f) probe(11,h)return h } func3(a′,b′,c′,d′) { e' = f′(a′,b′,c′,d′) probe(8,e′) returne′ }

Note that Func3 has a sub-function “f”, the inputs to which arevariables “a”-“d” and the output from which is variable “e”′,corresponding to variables “a”-“d” and “h”, respectively, of Func0.Sub-functions may also be probed using the bi-directional probingtechnique of the present invention.

By adding the probe instructions to the code under test, the software isobservable and therefore testable. Any type of variable (e.g., automatic(temporarily stored on the stack), global, static, etc.) or any storeddata may be probed so long as it is in the address space of the probe,as indicated by the variable in the probe instruction. It is alsopossible to probe function arguments using the bi-directional probe ofthe present invention. Probing the variables and function argumentsmakes it possible to further test the functionality of the software.Specifically, probing the variables and arguments of a function allowsadditional tests and test vectors to be developed based on the dataobtained.

An exemplary C-code version of the probe instructions can be seen in theblock of source code shown in Example 2 below. This example is providedto illustrate what an actual block of source code using embodiments ofthe invention might look like.

Example 2

// Calculates b * b + a int func0(int a, int b) { int d; int e; // areference to the variable is needed probe(2, &b); // so that the valuecould be changed d = func1(b); // this is the e = d + a; //functionality of func 0 probe(7, &e); return e; } int func1(int arg0) {int res; res = arg0 * arg0; //this is the functionality of func 1probe(8, &res); return res; }

The bi-directional probing technique of the present invention may beimplemented in any test system. FIG. 3 shows an exemplary test system300 for implementing the bidirectional probing technique. The testsystem 300 includes a tester 302 and a device under test 304 that are incommunication with each other. The tester 302 is a typical computer thathas a number of functional components, including a CPU 306, aninput/output interface unit 308, and a storage unit 310. Thesecomponents are well known to people of ordinary skill in the computerart and will therefore be described only briefly here. The CPU 306handles the execution of all software programs on the tester 302,including the operating system and any software running thereon. Theinterface unit 308 serves to interface the tester 302 to the deviceunder test 304, as well as any input/output devices (e.g., keyboard,mouse, display unit, printer, etc.) connected thereto. The storage unit310 provides temporary storage (e.g., RAM) and/or long-term storage(e.g., hard drive) for any software programs and/or data that may beneeded for the execution of the operating system and the softwarerunning on the tester 302.

Stored in the storage unit 310 are a number of software applications,including a software development tool 312. The software development tool312 operates in the same way and has many of the same features asexisting software development tools such as Code Composer Studio™ fromTexas Instruments and LabVIEW™ from National Instruments, or othersimilar software development tools. In accordance with embodiments ofthe invention, however, the software development tool 312 furtherincludes a probe control and analysis module 314. The probe control andanalysis module 314 is capable of controlling the bi-directional probingof any software being tested using the software development tool 312,and analyzing the data being probed. Specifically, the probe control andanalysis module 314 allows data to be captured from the code under test,injected into the code under test, or both, as determined by a user. Theprobe control and analysis module 314 also allows the user to generatetest vectors based on the data obtained and to inject the test vectorsback into the code under test. This makes it easier and more convenientfor the user to monitor and test the operation and reliability of thecode under test.

In the present embodiment, the code under test, including thebi-directional probe instructions, is executed on a separate unit,namely the device under test 304, that is in communication with thetester 302. The device under test 304, like the tester 302, is a typicalcomputer that has a number of functional components, including a CPU316, an input/output interface unit 318, and a storage unit 320. Thecomponents of the device under test 304 are similar in function to theircounterparts in the tester 302 and will therefore not be described here.The main point is that the code under test 322, including the probedsource code and the bi-directional probe instructions and implementationis stored and executed separately from the tester 302. (See theexemplary blocks of source code above for examples of probeinstructions.)

In some embodiments, however, the tester and the device under test areimplemented as a single, integrated test system that performs bothfunctions. FIG. 4 illustrates an example of such a test system 400. Theintegrated test system 400 has a number of functional components,including a CPU 402, an input/output interface 404, and a storage unit406. These components are similar to their counterparts described withrespect to FIG. 3, except that the storage unit 406 has both a softwaredevelopment tool 408 and a code under test 410 stored thereon. Thus, thetest system 400 preferably has sufficient storage and processingcapacity to execute both the software development tool 408 and the codeunder test 410 at the same time (i.e., multitasking). The softwaredevelopment tool 408 is essentially the same as the software developmenttool 312 described above, including a probe control and analysis module(not expressly shown). Likewise, the code under test 410 is essentiallythe same as the code under test 322 described above, including theprobed source code and probe instructions and implementation.

Execution of a bi-directional probe instruction is illustrated in theexemplary method 500 of FIG. 5, according to embodiments of theinvention. Such a method 500 is usually implemented on the device undertest that is executing the code under test, or an integrated testersystem in a multitasking environment. In the method 500, a block ofsource code that includes one or more probe instructions is in theprocess of being executed. At a certain point during the execution ofthe code, step 501, one of the probe instructions is encountered. At thenext step, step 502, a determination is made as to whether the probe hasbeen set in the probe mode. The particular mode is usually set by a uservia the tester from the software development tool 312, either as apreprogrammed command or manually, or a combination of both. If theanswer is yes, then at the third step 503, the data within the memory orstorage area indicated by the probe instruction is transmitted to thetest system where it can be monitored and analyzed as needed. If no,then the method 500 continues to the fourth step 504, where adetermination is made as to whether the probe has been set in injectmode. If the answer is yes, then at the fifth step 505, the data in thememory or storage area indicated by the probe instruction is revisedand/or replaced with new data received from test system using, forexample, a simple memory copy. If no, then the method 500 continues withexecution of the rest of the code under test.

Note that the method 500 described above is a simplified implementationof the probe instruction. In some embodiments, in addition todetermining the probing mode, the type of the data as well as its sizeare verified. It is also possible to probe more complicated to variablessuch as arrays and even variables that are not stored in a continuousstructure. In this manner, many types of data may be captured from, aswell as inserted into, the software while it is being tested.

While particular embodiments and applications of the present inventionhave been illustrated and described, it is to be understood that theinvention is not limited to the precise construction and compositionsdisclosed herein, and that modifications and variations may be made tothe foregoing without departing from the scope of the invention asdefined in the appended claims.

1. A method of testing software having a plurality of data variables andfunction arguments therein, comprising: executing the software;identifying an address location for at least one of the variables orarguments used by the software; outputting any data stored in theaddress location to a test system to thereby monitor the data; andinputting data from the test system into the address location to therebyreplace any data previously stored in the address location.
 2. Themethod according to claim 1, wherein the data inputted into the addresslocation is a revised version of the data outputted from the addresslocation.
 3. The method according to claim 1, wherein the data inputtedinto the address location is the same as the data outputted from theaddress location.
 4. The method according to claim 1, wherein the datainputted into the address location is generated based on the dataoutputted from the address location during a previous iteration.
 5. Themethod according to claim 1, wherein the data inputted into the addresslocation comprises a test vector of data generated based on dataoutputted from the address location during a previous iteration.
 6. Themethod according to claim 1, wherein the address location identifies astorage location in a computer memory.
 7. The method according to claim1, wherein the address location identifies a storage location in a harddrive.
 8. An apparatus for testing software having a plurality of datavariables and function arguments therein, comprising: a centralprocessing unit; a storage unit connected to the central processingunit, the storage unit storing computer readable instructions forinstructing the central processing unit to: execute the software;identify an address location for at least one of the variables orarguments used by the software; output any data stored m the addresslocation to the central processing unit to thereby monitor the data; andinput data from the central processing unit into the address location tothereby replace any data previously stored in the address location. 9.The apparatus according to claim 8, wherein the data inputted into theaddress location is a revised version of the data outputted from theaddress location.
 10. The apparatus according to claim 8, wherein thedata inputted into the address location is the same as the dataoutputted from the address location.
 11. The apparatus according toclaim 8, wherein the data inputted into the address location isgenerated based on data outputted from the address location during aprevious iteration.
 12. The apparatus according to claim 8, wherein thedata inputted into the storage area comprises a test vector of datagenerated based on data outputted from the address location during aprevious iteration.
 13. The apparatus according to claim 8, wherein theaddress location identifies a storage location in a computer memory. 14.The apparatus according to claim 8, wherein the address locationidentifies a storage location in a hard drive.
 15. A system for testingsoftware having a plurality of data variables and function argumentstherein, comprising: a device under test configured to execute thesoftware including one or more probe instructions in the software; atester connected to the device under test, the tester configured tocontrol the device under test so that when a probe instruction isexecuted, the device under test will: identify an address location forat least one of the variables or arguments used by the software; outputany data stored in the address location to the tester; and input datareceived from the tester into the address location.
 16. The systemaccording to claim 15, wherein the data inputted into the addresslocation is a revised version of the data outputted from the addresslocation.
 17. The system according to claim 15, wherein the datainputted into the address location is the same as the data outputtedfrom the address location.
 18. The system according to claim 15, whereinthe data inputted into the address location is generated based on dataoutputted from the address location during a previous iteration.
 19. Thesystem according to claim 15, wherein the data inputted into the storagearea comprises a test vector of data generated based on data outputtedfrom the address location during a previous iteration.
 20. The systemaccording to claim 15, wherein the address location identifies a storagelocation in a computer memory.
 21. The system according to claim 15,wherein the address location identifies a storage location in a harddrive.
 22. The system according to claim 15, wherein the tester and thedevice under test reside in a single unit.